Research on Scientific Journals: Implications for Editors and Authors

A review of editorial policies of leading journals and of research relevant to scientific journals revealed conflicts between 'science' and 'scientists.' Owing to these conflicts, papers are often weak on objectivity and replicability. Furthermore, papers often fall short on importance, competence, intelligibility, or efficiency. Suggestions were made for editorial policies such as: (1) structured guidelines for referees, (2) open peer review, (3) blind reviews, and (4) full disclosure of data and method. Of major importance, an author's "Note to Referees" (describing the hypotheses and design, but not the results) was suggested to improve the objectivity of the ratings of importance and competence. Also, recommendations are made to authors for improving contributions to science (such as the use of multiple hypotheses) and for promoting their careers (such as using complex methods and obtuse writing). Comments Postprint version. Published in Journal of Forecasting, Volume 1, Issue 1, 1982, pages 83-104. Publisher URL: http://dx.doi.org/10.1002/for.3980010109 This journal article is available at ScholarlyCommons: http://repository.upenn.edu/marketing_papers/113 Published in Journal of Forecasting, 1, 1982, 83-104. Research on Scientific Journals: Implications for Editors and Authors J. Scott Armstrong The Wharton School, University of Pennsylvania Abstract A review of editorial policies of leading journals and of research relevant to scientific journals revealed conflicts between 'science' and 'scientists.” Owing to these conflicts, papers are often weak on objectivity and replicability. Furthermore, papers often fall short on importance, competence, intelligibility, or efficiency. Suggestions were made for editorial policies such as: (1) structured guidelines for referees, (2) open peer review, (3) blind reviews, and (4) full disclosure of data and method. Of major importance, an author's “Note to Referees” (describing the hypotheses and design, but not the results) was suggested to improve the objectivity of the ratings of importance and competence. Also, recommendations are made to authors for improving contributions to science (such as the use of multiple hypotheses) and for promoting their careers (such as using complex methods and obtuse writing).A review of editorial policies of leading journals and of research relevant to scientific journals revealed conflicts between 'science' and 'scientists.” Owing to these conflicts, papers are often weak on objectivity and replicability. Furthermore, papers often fall short on importance, competence, intelligibility, or efficiency. Suggestions were made for editorial policies such as: (1) structured guidelines for referees, (2) open peer review, (3) blind reviews, and (4) full disclosure of data and method. Of major importance, an author's “Note to Referees” (describing the hypotheses and design, but not the results) was suggested to improve the objectivity of the ratings of importance and competence. Also, recommendations are made to authors for improving contributions to science (such as the use of multiple hypotheses) and for promoting their careers (such as using complex methods and obtuse writing). The major purpose of the paper is to suggest improved policies for scientific journals. Many of these policies are already used by journals. This paper is also directed to authors. While the goals of authors and editors often agree, on some points they conflict. Recognition of the conflicts may help authors to achieve better their goals, whether they relate to career advancement or to scientific contributions. Scientific journals provide a foundation for scientific progress. They aid in the communication of knowledge and provide an historical record. In Levitan’s (1979) survey of biomedical scientists, 85 percent ranked 'the journal' as the most frequently used information source for keeping up to date. Since 1960, the number of journals has increased by about 2 percent per year, a bit slower than the growth in the number of scientists (King et al., 1978; Garvey and Griffith, 1970). This growth has occurred despite claims by scientists that the overall quality of papers is poor. For example, in a convenience sample of 15 physicists, all thought that the vast majority of papers in the journals that they read were of poor quality or little significance (Mulkay and Williams, 1971). This paper reviews the current state of knowledge about scientific journals. It draws upon the editorial policies of many journals. Particularly useful in this respect were the guidelines prepared in 1976 for the editor of the Journal of Personality and Social Psychology (Greenwald, 1976), and the survey of authors and readers of the American Psychological Association journals (Brackbill and Korten, 1970). I reviewed empirical research on scientific papers. Most of this research was from the social sciences, but some was from the natural sciences. Experts from a variety of fields also assisted me. The 1 These included Stephen Brush (history of natural sciences), James Cameron (publishing), Robert Carbone (operations research), Robert Ferber (survey research), Robert Fildes (operations research), Baruch Fischhoff (psychology), Nigel Gilbert (sociology of science), Robin Hogarth (management), David Horrobin (medicine), Ed Lusk (operations research), Albert Madansky (statistics), Spyros Makridakis (forecasting), and Bill Remus [continued on next page]